Biocybernetic Adaptation Strategies: Machine awareness of human state for improved operational performance

Human operators interacting with machines or computers continually adapt to the needs of the system ideally resulting in optimal performance. In some cases, however, deteriorated performance is an outcome. Adaptation to the situation is a strength expected of the human operator which is often accomplished by the human through self-regulation of mental state. Adaptation is at the core of the human operator’s activity, and research has demonstrated that the implementation of a feedback loop can enhance this natural skill to improve training and human/machine interaction. Biocybernetic adaptation involves a “loop upon a loop,” which may be visualized as a superimposed loop which senses a physiological signal and influences the operator’s task at some point. Biocybernetic adaptation in, for example, physiologically adaptive automation employs the “steering” sense of “cybernetic,” and serves a transitory adaptive purpose – to better serve the human operator by more fully representing their responses to the system. The adaptation process usually makes use of an assessment of transient cognitive state to steer a functional aspect of a system that is external to the operator’s physiology from which the state assessment is derived. Therefore, the objective of this paper is to detail the structure of biocybernetic systems regarding the level of engagement of interest for adaptive systems, their processing pipeline, and the adaptation strategies employed for training purposes, in an effort to pave the way towards machine awareness of human state for self-regulation and improved operational performance

Disruption in neural phase synchrony is related to identification of inattentional deafness in real-world setting

Individuals often have reduced ability to hear alarms in real world situations (e.g., anesthesia monitoring, flying airplanes) when attention is focused on another task, sometimes with devastating consequences. This phenomenon is called inattentional deafness and usually occurs under critical high workload conditions. It is difficult to simulate the critical nature of these tasks in the laboratory. In this study, dry electroencephalography is used to investigate inattentional deafness in real flight while piloting an airplane. The pilots participating in the experiment responded to audio alarms while experiencing critical high workload situations. It was found that missed relative to detected alarms were marked by reduced stimulus evoked phase synchrony in theta and alpha frequencies (6–14 Hz) from 120 to 230 ms poststimulus onset. Correlation of alarm detection performance with intertrial coherence measures of neural phase synchrony showed different frequency and time ranges for detected and missed alarms. These results are consistent with selective attentional processes actively disrupting oscillatory coherence in sensory networks not involved with the primary task (piloting in this case) under critical high load conditions. This hypothesis is corroborated by analyses of flight parameters showing greater maneuvering associated with difficult phases of flight occurring during missed alarms. Our results suggest modulation of neural oscillation is a general mechanism of attention utilizing enhancement of phase synchrony to sharpen alarm perception during successful divided attention, and disruption of phase synchrony in brain networks when attentional demands of the primary task are great, such as in the case of inattentional deafness

Predicting video game players’ fun from physiological and behavioural data : one algorithm does not fit all

Finding a physiological signature of a player’s fun is a goal yet to be achieved in the field of adaptive gaming. The research presented in this paper tackles this issue by gathering physiological, behavioural and self-report data from over 200 participants who played off-the-shelf video games from the Assassin’s Creed series within a minimally invasive laboratory environment. By leveraging machine learning techniques the prediction of the player’s fun from its physiological and behavioural markers becomes a possibility. They provide clues as to which signals are the most relevant in establishing a physiological signature of the fun factor by providing an important score based on the predictive power of each signal. Identifying those markers and their impact will prove crucial in the development of adaptive video games. Adaptive games tailor their gameplay to the affective state of a player in order to deliver the optimal gaming experience. Indeed, an adaptive video game needs a continuous reading of the fun level to be able to respond to these changing fun levels in real time. While the predictive power of the presented classifier remains limited with a gain in the F1 score of 15% against random chance, it brings insight as to which physiological features might be the most informative for further analysis and discuss means by which low accuracy classification could still improve gaming experience

Mixed-Initiative Human-Automated Agents Teaming: Towards a Flexible Cooperation Framework

The recent progress in robotics and artificial intelligence raises the question of the efficient artificial agents interaction with humans. For instance, artificial intelligence has achieved technical advances in perception and decision making in several domains ranging from games to a variety of operational situations, (e.g. face recognition [51] and firefighting missions [23]). Such advanced automated systems still depend on human operators as far as complex tactical, legal or ethical decisions are concerned. Usually the human is considered as an ideal agent, that is able to take control in case of automated (artificial) agent's limit range of action or even failure (e.g embedded sensor failures or low confidence in identification tasks). However, this approach needs to be revised as revealed by several critical industrial events (e.g. aviation and nuclear power-plant) that were due to conflicts between humans and complex automated system [13]. In this context, this paper reviews some of our previous works related to human-automated agents interaction driving systems. More specifically, a mixed-initiative cooperation framework that considers agents' non-deterministic actions effects and inaccuracies about the human operator state estimation. This framework has demonstrated convincing results being a promising venue for enhancing human-automated agent(s) teaming

A Neuroergonomics Approach to Measure Pilot’s Cognitive Incapacitation in the Real World with EEG

Mental overload and mental fatigue are two degraded cognitive states that are known to promote cognitive incapacitation. We adopted a neuroergonomics approach to investigate these states that remain difficult to induce under la-boratory settings thus impeding their measurement. Two experiments were conducted under real flight conditions to respectively measure the electro-physiological correlates of mental fatigue and mental overload with a 32 chan-nel-dry EEG system. Our findings revealed that the occurrence of mental fatigue was related to higher theta and alpha band power. Mental overload was associ-ated with higher beta band power over frontal sites. We performed single trial classification to detect mental fatigue and over-load states. Classification accu-racy reached 76.9% and 89.1%, respectively, in discriminating mental fatigue vs. no fatigue and mental overload vs. low-high load. These preliminary results provide evidence for the feasibility of detecting neural correlates of cognitive fatigue and load during real flight conditions and provide promising perspec-tives on the implementation of neuroadaptive technology especially in the con-text of single pilot-operation

Building Speech Recognition Systems for Language Documentation: The CoEDL Endangered Language Pipeline and Inference System (ELPIS)

Machine learning has revolutionised speech technologies for major world languages, but these technologies have generally not been available for the roughly 4,000 languages with populations of fewer than 10,000 speakers. This paper describes the development of Elpis, a pipeline which language documentation workers with minimal computational experience can use to build their own speech recognition models, resulting in models being built for 16 languages from the Asia-Pacific region. Elpis puts machine learning speech technologies within reach of people working with languages with scarce data, in a scalable way. This is impactful since it enables language communities to cross the digital divide, and speeds up language documentation. Complete automation of the process is not feasible for languages with small quantities of data and potentially large vocabularies. Hence our goal is not full automation, but rather to make a practical and effective workflow that integrates machine learning technologies

Performance and brain activity during a spatial working memory task: application to pilot candidate selection

International audienceFor 18 ab initio airline pilots, we assessed the possibility of predicting flight simulator performance with the performance and the prefrontal activity measured during a spatial working memory (SWM) task. Behavioral results revealed that a better control of the aircraft altitude in the flight simulator was correlated with better strategy during the SWM task. In addition, neuroimaging results suggested that participants that recruited more neural resources during the SWM task were more likely to accurately control their aircraft. Taken together, our results emphasized that spatial working memory and the underlying neural circuitries are important for piloting. Ultimately, SWM tasks may be included in pilot selection tests as it seems to be a good predictor of flight performance